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ABSTRACT 

This report summarizes the development of a 
commercially available program in computational skills which makes 
extensive use of computer capabilities. The first section describes 
the organization, content, and cost of the mater:tals, and the 
procedures for using them. The key personnel, and the organizational 
and funding sources are then named. Three phases in the development 
process are next identified; (1) informal tryouts, (2) program 
development using the APL/360 programming language, and (3) 
conversion to Coursewriter III language. Formative evaluation during 
the second phase and summative evaluation after the third phase are 
described in some detail. Final sections of the report outline the 
marketing procedures for the product, its degree of adoption, 
possible future developments, and four crucial decisions in its 
developmental history. (MM) 
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PREFACE 

This product development report is one of 21 such reports, each dealing 
with the developmental history of a recent educational product. A list of the 
21 products, and the agencies responsible for their development, is contained 
in Appendix A to this report. The study, of which this report is a component, 
was supported by U.S. Office of Education Contract No. OCC-0-70-4892, entitled 
"The Evaluation of the Impact of Educational Research and Development Products." 
The overall project was designed to examine the process of ilevelopment of 
"successful educational products." 

This report represents a relatively unique attempt to document what 
occurred in the development of a recent educational product that appears to 
have potential impact. The report is based upon published materials, docu- 
ments in the files of the developing agency, and interviews with staff who 
were involved in the development of the product. A draft of each study was 
reviewed by the developer's staff. Generally, their suggestions for revisions 
were incorporated into the text; however, complete responsibility for inter- 
pretations concerning any facet of development, evaluation, and diffusion 
rests with the authors of this report. 

Although awareness of the full impact of the studs requires reading both 
the individual product development reports and the separate final report, each 
study may be read individually. For a quick overview of essential events in 
the product history, the reader is referred to those sections of the report 
containing the flow chart and the critical decision record. 

The final report contains: a complete discussion of the procedures and 

the selection criteria used to identify exemplary educational products; gener- 
alizations drawn from the 21 product development case studies; a comparison of 
these generalizations with hypotheses currently existing in the literature 
regarding the processes of innovation and change; and the identification of 
some proposed data sources through which the U.S. Office of Education could 
monitor the impact of developing products. The final report also includes a 
detailed outline of the search procedures and the information sought for each 
case report. 

Permanent project staff consisted ol Calvin E. Wr.ght, Principal 
Investigator; Jack J. Crawford, Project Director; Daniel W. Kratochvil, Research 
Scientist; and Carolyn A. Morrow, Administrative Assistant. In addition, other 
staff who assisted in the preparation of individual product reports are identi- 
fied on the appropriate title pages. The Project Monitor was Dr. Alice Y. 

Scates of the USOE Office of Program Planning and Evaluation. 

Sincere gratitude is extended to those overburdened staff members of the 
21 product development studies who courteously and freely gave their time so 
that we might present a detailed and relatively accurate picture of the events 
in the development of some exemplary educational research and development pro- 
ducts. If we have chronicled a just and moderately com.ilete account of the 
birth of these products and the hard work that spawned them, credit lies with 
those staff members of each product development team who ransacked memory and 
files to recreate history. 
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PRODUCT DESCRIPTION 



Product Characteristics 

Name 

The Arithmetic Proficiency Training Program (APTP). 

Developer 

Science Research Associates, Inc., a wholly owned subsidiary of IBM. 
Distributor 

~ I 

Science Research Associates, Inc. 

Focus 

The Arithmetic Proficiency Training Program is a computer-assisted 
instructional program for supplementary use in the development of the compu- 
tational skills of elementary arithmetic. It can be used with any elementary 
mathematics program — traditional, transitional, or modern. 

Grade Level 

The Arithmetic Proficiency Training Program is designed for grades one 
through eight and can be used remedlally at any older age as well. 

Target Population 

The target population of APTP consists of all students in need of addi- 
tional proficiency in computational skills. 

Rationale for Product 

Long Range Goals of Product 

The long range goal of APTP is to Individualize the learning of computa- 
tional skills by students. By helping students master such skills through 
drill and practice and at their own level and their own rate, APTP is designed 
to free the teacher from the supervision of practice so that he can develop 
concepts, present ideas, and facilitate the student's understanding of arith- 
metic. Due to the ease of revision and expansion of a computer program (as 
contrasted with the difficulty and cost of revising a textbook series), the 
developers pirn on-going improvement and enlargement of the program on the 
basis of feedback from user schools and of new ideas to be built in. 
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Objectives of Product 



The objectives of APTP are: 

1. To help the student achieve mastery of the computational 
skills customarily Introduced in grades 1-6. 

2. To provide the student personalized practice after indi- 
vidualized diagnosis has shown where practice is most 
appropriate . 

3. To develop the student's speed and accuracy in computing, 
through sequences of timed practice problems. 

The computational skills cover five areas and include 29 skills called mole- 
cules. In turn, each molecule is analyzed into a number of subskills called 
atomic skills. The atomic skills, which are very specific (e.g., finding the 
sum of four one-digit numbers where the sum is greater than or equal to 20) , 
are organized in hierarchies. Each atomic skill with its associated mastery 
criteria constitutes a behavioral objective. 

Philosophy and Theories Supporting Product 

With regard to mathematics education, a fundamental assumption of APTP 
is that the student "must be able to ^ things, not just understand them." 

That is, to be fully competent in arithmetic, the student must have computational 
skills as well as an understanding of basic arithmetic concepts. Thus, students 
need practice in making computations. However, this practice should be indi- 
vidualized. 

Individualization means that the student should work at his own level and 
at his own pace and that he should receive however much practice he needs and 
only that much. To accomplish the task of finding each student's appropriate 
level requires testing and, more specifically, diagnostic testing. Diagnostic 
testing is accomplished in the program by use of Gagne-type skill analyses and 
hierarchies. If a student cannot meet mastery criteria on a particular skill, 
the program searches through the prerequisite skills in the hierarchy until it 
locates some skill where the student cannot meet criteria but for which he has 
demonstrated mastery on the immediately prerequisite skills. At this point, 
the program shifts from its test mode into the practice mode and the student 
receives however many practice problems he needs to meet criteria. 

There is considerable influence of Skinnerian ideas clearly visible in 
the program: reinforcement by immediate feedback is considered an essential 



feature of the program; progress through very small steps is also an Important 
feature; creating a situation where each student will experience success at 
his level Is also Important. 

Description of Materials 

Organization and Format of Materials 

Physically, the Arithmetic Proficiency Training Program consists of two 
computer tapes and a variety of supporting guides and manuals. These are 
divided Into a set of licensed materials (available only as a package leased 
by SRA to the user) and a set of unlicensed materials, which may be purchased 
separately. The licensed materials includ ^ the Basic Tape (consisting of the 
Coursewrlter III statements and the Assembly Language functions needed to 
actually run the program on a computer), the Optional Tape (source card images, 
assembly listings, and flow charts), and the various operating instructions 
(Application Directory, Systems Manual, and Operator's Manual). These last 
three provide instructions to personnel operating the computer , so that they may 
understand the inner workings of the computer program. 

The unlicensed materials include the Teacher's Guide, the Proctor's Guide, 
and the Student Record Book. These are the materials of concern to people in 
the school. The Teacher's Guide describes the program and Its rationale, gives 
all the molecular skill maps (such as the one In Figure 1 on page 6) and 
detailed specifications of all the atomic skills (together with the "standard" 
settings for mastery speeds and block sizes), describes the several options 
for use of the program, and generally informs the teacher about how to use the 
program. The Proctor's Guide gives very specific and detailed instructions to 
the proctor, who is assumed to be a paraprofesslonal and whose functions Include 
loading paper Into the terminals, changing typewriter ribbons, helping students 
vith the operation of the keyboard, etc. The Student Record Book consists of 
skill maps of each molecule; it provides the student with a graphic guide to 
the molecules so that he can keep track of his progress. 

Content of Materials 

The Basic computer tape for the Arithmetic Proficiency Training Program 
\consists of (1) basic files of computational algorithms and the number genera- 
tors which provide the desired numbers In the problems; and (2) the computer 
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instructions for processing Che test and practice interactions of the student 
as well as making both momentary and cumulative reports of these interactions. 

Educationally, Che material is divided into Che following 29 molecules 
which can be grouped into the five indicated groups. 

Operations on Whole Numbers 

Addition of Whole Numbers 
Subtraction of Whole Numbers 
Multiplication of Whole Numbers 
Division of Whole Numbers 

Operations on Fractions 

Fractions — Lowest Terms 
MultlpllcsiClon of Fractions 
Division of Fractions 
Fractions — Higher Terms 
Fractions — Common Denominator 
Addition of Fractions 
Subtraction of Fractions 
Comparison of Fractions 

Operations on Decimals 

Addition of Decimals 

Subtraction of Decimals 

Multiplication of Decimals 

Division of Decimals 

Conversion of Decimals to Fractions 

Conversion of Fractions to Decimals 

Operations of Mixed Numbers 

Conversion of Fractions to Mixed Numbers or 

Whole Numbers 

Addition of Mixed Numbers 

Conversion of Whole Numbers and Mixed Numbers 
to Fractions 

Subtraction of Mixed Numbers 
Multiplication of Mixed Numbers 
Division of Mixed Numbers 

Operations on Percents 

Conversion of Decimals and Fractions to Percents 
Conversion of Percents to Decimals and Fractions 
Finding Percentage from Rate and Base 
Finding Rate from Base and Percentage 
Finding Base from Rate and Percentage 
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Each molecule (e.g., Finding Base from Rate and Percentage) is a complete 
program in Itself and has been minutely subdivided into smaller skills called 
atomic skills. An atomic skill represents one idea or one Shade of complexity 
in the molecular operation. For exsunple, adding three one-digit numbers with 
a sum less than ten is a different and less advanced atomic skill than adding 
three one-dlglt numbers with a sum greater than or equal to ten. For each 
molecule, the atomic skills are arranged in a learning hierarchy. This Mer- 
archy, or interrelationship between atomic skills, is shown in a skill map, 
such as the one in Figure 1 on the following page, for the addition of whole 
numbers. Each circle indicates an atomic skill and each arrow from one skill 
to another shows that mastery of the first skill is prerequisite to mastery of 
the second. 

The program consists of 409 atomic skills like the 32 shown in Figure 1. 
Associated with each of these 409 atomic skills is a "block size" which, among 
other things, determines the minimum number of practice problems necessary to 
meet the mastery criteria; as problems Increase in complexity, the block size 
decreases. For example, with the "standard" setting of 80% accuracy in the 
practice mode, the student must get four out of five problems right in Skill 32 
of addition of whole numbers but must get 16 out of 20 right in the much 
simpler Skill 3, in order to meet criteria. With the minor exception of a few 
skills, the problems are "randomly" generated or selected by the computer pro- 
gram, within the very tlgjht constraints of the skill specifications. This 
programming technique has the virtue of making available a very large nximber 
of different problems wltho._t the labor of writing these problems "by hand." 

The second level of programming turns the student's interaction with the areas 
and skills Into a learning experience based upon placement and practice func- 
tions, with immediate feedback on accuracy and speed. This level of programming 
is discussed more extensively below, under Learner Activities. 

I 

Cost of Materials to User 

AFTP is licensed by SRA to customers at an annual use charge of $9,000. 
Payment of this charge entitles the customer to use the computer program on one 
central processing unit. An installation and training service by SRA is pro- 
vided for an additional fee, at the customer's option. 

Included in this $9,000 are licensed materials (l.e., tapes and instruc- 
tions) and five copies each of the unlicensed materials (l.e.. Teacher's Guide, 
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Figure 1* Skills Map for Addition of Whole Nximbers 
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Proctor's 


Guide and Student Record). 


Additional copies 


of the unlicensed 


materials 


are available at the following prices: 








List 


Net 




Teacher's Guide 


$10.00 


$7.50 




Proctor's Guide 


6.67 


5.00 




Student Record 


1.00 


.75 


The above 


prices are based on SRA's 


1971 catalog and are subject to change. 




Procedures 


for Using Product 





Learner Activities 

In APTP, the student works at a computer terminal that is similar to an 
electric typewriter. Examples of the interaction between the student and the 
computer program are shown in Figure 2 on the following two pages. The arrow 
in the left margin (which would not ordinarily be printed) indicates the 
typing done by the student; the rest is printed by the computer. The student 
spends about 20 minutes a day, or whatever length of time is considered 
appropriate, at the terminal, through which he communicates with the computer 
program. The student is first introduced to the APTP program by the teacher 
who gives him an overview of the program, the keyboard training, the computer, 
the role of the proctor, and her role as the teacher. 

The initial segment in the program is designed to teach the student how 
to use the terminal. Like all other parts of APTP, this one is a dialogue 
between the student and the computer program. After every response the student 
makes, he receives a reply telling him how well he did (l.e., his accuracy 
and speed). While at the terminal, the student is led through exercises in 
typing numbers and performing several keyboard functions. He is recycled 
through these exercises until he has mastered the skills. Then he is allowed 
to begin his first arithmetic molecule. 

The student's work on each arithmetic molecule is divided into two phases- 
placement and development. The placement phase is designed to determine the 
student's current level of ability in a molecule, while the development phase 
is designed to take him from his current level to mastery of the most diffi- 
cult skill at the top of the molecule. In the placement phase, the student 
begins with a test in the middle of a molecule and moves up or down in 
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Figure 2. Examples of Che Interact Ion 
between the student and the computer 

SKILL NUMBER 5 - TEST 



SAMPLE PROBLEM: 




14/1 

MASTERY SPEED: 7 SECONDS. 

******* ******************************* 




9/6 

No. That's not right. 
************************************** 




38/3 
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No. That's not right. 

************************************** 

i 
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You have not mastered this skill yet. 
********************************************************** 
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************************************** 
SKILL NUMBER 5 - PRACTICE 



SAMPLE PROBLEM; 




14/1 

MASTERY SPEED: 7 SECONDS. 

************************************** 



li + 26 ^ , 

3 3 

35/3 

No. That's not right. 

19 + 26 = ? 

45 

OK. Now try the original problem. 

3 ^ 3 ' 

45/3 

Yes. But your answer must be reduced to lowest terms. 

19 + 26 ^ , 

3^3 

15/1 

Right, but too slow. 
************************************** 





9/1 

Right, but too slow. 

(17 seconds) 

************************************** 
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PERFECT I 
(5 seconds) 

************************************** 
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successively smaller steps to other tests on atomic skills until he arrives 
at his appropriate level. The direction In which the student moves from test 
to test depends on whether he passes or falls. To pass a test for an atomic 
skill, he must answer 80% of the problems correctly and within the mastery 
speed specified for the atomic skill. Sample placement sequences for two 
students in the subtraction molecule have been traced In Figure 3 below. 

Each student took four tests; one found his level at atomic skill 9 and the 
other at 17. 

Figure 3. Sample Placement Sequence 
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4 if PASS, up 4 
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The student enters the developmental phase of the molecule after he has 
finished the last test in the placement phase. A flow chart of the steps in 
the developmental phase appears in Figure 4. A tentative list of skills, 
which Includes the skills from the student's placement level to the top of 
the molecule and that the student must master to complete the molecule, is 
set up for the student (step 1). If the student passed the highest skill 
during placement, there would be no skills on the list (step 2) and he would 
be sent on to the next molecule (step 3). Next, the program determines if the 
student has been tested on the first skill on the list (step 4) and whether he 
passed or failed it (step 5). If he failed, a prerequisite testing system 
locates deficiencies related, to the failed skill and adds these to his list of 
skills (step 7). Steps 5, 6. 7, and 8 are repeated until the program finds a 
skill that the student has failed but that has prerequisites that he can pass , 
or until there are no more direct prerequisites in the molecule. Then the 
student is given practice in the skills that he has not mastered, starting 
with the one lowest in the hierarchy (step 9). Practice problems are presented 
to the student in blocks of varying length, depending on the complexity of the 
atomic skill. The student is cycled through these blocks until he has mastered 
one. When he has, the program removes the atomic skill from the student's list 
(step 2) , sends the student out of the molecule if there are not any left 
(step 3), or begins the cycle again (step 4), if there are some left. 

What has been described are the typical student activities under the basic 
"X" option of operation. Additionally, there are three other options for use 
of the program. Under the "M" option, the teacher has greater flexibility in 
determining the course of the student's experience with APTP, in that she may 
make adjustments of such parameters as mastery speed, block size, and accuracy 
criteria. Under the "S" option, the teacher is allowed to select any of the 
29 molecules, any one of the atomic skills in that molecule, and to specify 
either test or practice mode; no opportunity for adjustments of parameters 
occurs in this option, as it does in the "M" option. In the spiral curriculum 
option, the student follows a spiral curriculum of the teacher's choice through 
any or all of the 29 arithmetic molecules of the program. This option is 
illustrated by the example in Figure 5. The spiral in Figure 5 is one in 
which the student would first demonstrate mastery of low~level skills in 
Addition of Whole Numbers, would then go on to low-level skills in Subtraction 
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Figure 4. Diagram of Steps in Developmental Phase 
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of Whole Numbers, and then to low-level skills In Multiplication of Whole 
Numbers. This would constitute "Orbit A" of the spiral. Then, the student 
continues on to Orbit B where he successively demonstrates mastery of the 
next higher level of skills in Addition, in Subtraction, and in Multiplica- 
tion and low-level skills in Division of Whole Numbers. The student then 
goes on to Orbit C. This does not require any teacher intervention. This 
option is claimed to be especially effective in developmental use of the pro- 
gram with children in the early grades, as contrasted to remedial use with 
older students. 

When a student completes a session at the terminal, a record of his 
performance for the session is printed out. This provides the student with 
an overview of his progress for that day. To keep a cumulative record of 
his progress, the student keeps track of his progress in the APTP Student 
Record Book by checking off completed skills that are mapped for each mole- 
cule in the book. 

Teacher Activities 

The teacher registers the student in the APTP program; Introduces 
him to the program; controls, if she chooses, some features of the develop- 
mental phase; responds to the student when the program generates a "teacher 
call"; and discusses with the student his dally progress. The caretaking 
of the machines and records Is delegated to a proctor who can be any respon- 
sible adult (e.g., a teaching assistant, a parent or a college student). 

When introducing the student to the program, the teacher needs to explal 
the purpose of the program and the student's role in it, as well as her own 
role in it. The Teacher's Guide provides information pertaining to this 
teacher activity as well as information on the other teacher activities. The 
teacher might choose to control some of the features of the developmental 
phase instead of having the program operate them. She can: 

1. List the atomic skills the student should use in a molecule. 

2. Specify test or practice mode for any atomic skill. 

3. Alter the block size, the mastery speed, or the accuracy 
criteria. 
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Figure 5. The Spiral Curriculum Option 
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If the student does not Improve from block to block In a practice sequence, 
the program generates a "teacher call," which tells the student to see his 
teacher before continuing. The teacher then tries to Isolate and remedy 
the difficulty by analyzing the student's printout. Also, she can review the 
student's progress by examining his printout with him at the end of each day 
or week. 

The Computer Related Instructional Systems Center offers consulting 
services to teachers who use the AFTP program. Instruction Includes use of 
the Teacher's Guide and supervised practice on the system as desired. 

Provisions for Parent /Community Involvement 

No special effort was made to Involve parents or community in the use 
of APTP. However, the student does have his own printout available at the 
end of each of his Interactions with the computer. Furthermore, his record 
book provides a means for him to know and report on his activities In master- 
ing computational skills. The developers, therefore, report that the 
Arithmetic Proficiency Training Program ordinarily receives enthusiastic 
attention from parents because it provides them, with a comprehensive and 
meaningful means of understanding what their child Is doing and why. It 
Is also possible for parents to serve as proctors, as mentioned under Teacher 
Activities. 

Special Physical Facilities or Equipment 

As Indicated earlier, access to a computer Is needed. It should be under- 
stood that the computer need not be in the school and that the computer will 
probably be used for administrative data processing such as payroll processing 
and class scheduling. Computer terminals are needed and a special room Is 
recommended for these terminals, since experience with terminals In regular 
classrooms has thus far not been found satisfactory. In addition. If the 
computer Is appreciably remote from the school, there Is need for telephone 
connections between computer and terminals. 

Recommended Assessment Techniques for Users 

AFTP deliberately blurs the usual distinction between test and practice, 
because of the developers' belief that student performance can be measured In 
much the same way In either a test situation or a practice situation. Thus 
the same kinds of problems are given In both test and practice modes; the 
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major distinctions between the test mode and the practice mode are: (1) in 

the practice mode, the student may be given several chances to get the correct 
answer for a given problem while in the test mode he has only one chance; 

(2) a test session on a given atomic skill Is generally shorter than a prac- 
tice session on that skill. 

AFTP thus provides a criterion-referenced assessment procedure keyed 
directly to each atomic skill in the hierarchy of skills in a molecule. In 
a school system making extensive use of criterion-referenced assessment, 
there would be little or no need for assessment techniques other than those 
provided by the record-keeping of AFTP. The developers say, however, that most 
schools continue to evaluate programs of instruction on the basis of nationally 
standardized achievement tests and they feel that there is much value in care- 
fully controlled evaluation plans with pre— and posttesting with such tests. 

In fact, they believe that such evaluation systems may be the only practical 
way by which school people will be able to decide whether there is signifi- 
cant value in computer-assisted instruction. 

Th^ warn, however, that their experience shows considerable danger that 
standardized achievement tests may lack adequate sensitivity at the lower end 
to reveal important gains made by students who are seriously below grade level. 
For example, many disadvantaged 7th-graders may be sadly lacking in mastery of 
the multiplication table but the tests usually administered at that grade 
level may be inadequate to reveal such lack or to shov; gains achieved by special 
programs. 



ORIGINS 
Key Personnel 

The key personnel involved in the development of APTP contributed four 
types of expertise: management, subject matter and curriculum design, computer 

programming, and research and evaluation. Don Mitchell was director when the 
project began, although Its conception and early planning occurred under John 
Lawrence, his predecessor and founder of the Computer Related Instructional 
Systems Center (CRIS) . Noel Capettlnl became director in late 1968 and 
managed APTP from mid-development through completion of the product version. 
Bernard Jeltema is the present director and manages the continued development 
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and dissemination of APTP. Each of these directors was an SRA or IBM employee 
before becoming director of the Center. Their previous experiences involved 
computer related activities, teaching, and research and development. 

Robert Fouch provided the subject matter and curriculum design expertise; 
he might be described as the principal architect of the program. He had had 
extensive experience in teaching and in teacher education and had been 
Director of Mathematics for SRA's Materials of Instruction Division before 
working on APTP. 

Ed Foster and others supplied the computer programming skills. In 
addition to these computer skills, Foster's background included teaching 
experience. 

Robert Rehula headed up the evaluation program for APTP. His background 
was in learning theory, and he had worked on the SRA Reading Laboratories. 

Thus, all key personnel were previously involved in work at SRA or IBM before 
working on the development of APTP . 

Sources and Evolution of Ideas for Product 

During Science Research Associates' early years, extending back to 1938, 
their focus was on testing and guidance. The first SPA Reading Laboratory 
was published in 1958. The single most important aspect, which came to 
influence development of many other instructional products (e.g., APTP), was 
that the Laboratory provided for individualization of instruction in the 
classroom. According to the developers, this reading program was so success- 
ful that "it just took off and flew." Because of this success SRA made the 
decision in 1959 to branch out into other subject matter areas. In 1959-60 
people were hired to start efforts in social studies, mathematics, and science. 

The social studies people got off to a quick start by designing materials 
very much in the same nature as the Reading Laboratory. Robert Fouch was 
brought in to develop similar programs in mathematics. He pointed out that 
even though the goals set for his department were very clear, the initial 
attempts at planning individualized materials in mathematics resulted in some- 
thing that was physically unmanageable. Typically, they tried to cover too 
large an area. For several years, the department put most of its efforts into 
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production of math materials In more conventional media and format (e.g. , 
the Greater Cleveland Mathematics Program.) Eventually, when they decided 
to narrow things down to just the computational aspects of elementary mathe- 
matics, they produced a Computational Skills Development Kit (CSDK) which 
was supplementary and very similar to the Reading Laboratories, both physically 
and structurally. Other individualized programs, each with a narrow scope, 
were also produced. CSDK had the virtues of providing individualized test- 
ing and placement into a linear sequence of practice work; there were pro- 
visions for quick (but not immediate) feedback to the student on his practice 
work. Fouch explained that CSDK was something of a minor breakthrough and 
that, although it was quite successful, it had many limitations resulting 
from the paper-and-ink medium used: provisions for testing were limited, 

feedback was not really immediate, the number of different practice problems 
was severely limited, the program was essentially a linear one, etc. CSDK 
and APTP (as later developed) might be said to have almost exactly the same 
goals; however, the limitations imposed by the paper-and-ink medium vanished 
when the potential of the computer was used in the development of APTP. 

Funding for Product Development 

Funding arrangements for development and testing of the Arithmetic 
Proficiency Training Program were entirely internal to IBM and Science Research 
Associates (which is a wholly owned subsidiary of IBM). Even in the Chicago 
field test (described later), the Chicago schools paid only for telephone 
costs; they did, of course, contribute greatly in terms of teacher cooperation 
but this was achieved without any special funding. 

The cost of developing the Arithmetic Proficiency Training Program is 
proprietary information and could not be obtained. However, a rough estimate 
of the man-years of effort required to develop APTP would be about 20 man- 
years. At about $40,000 to $50,000 per man-year (including all support cost, 
especially costs of equipment), an estimate of IBM's investment would be a 
million dollars. Fouch insists, however, that such an estimate is not at all 
representative of present CAI development costs and that a program of like 
size could be developed for much less with today's "know-how." 
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PRODUCT DEVELOPMENT 
Management and Organization 

The Computer Related Instructional Systems Center of Science Research 
Associates is the developer of APTP. IBM purchased Science Research Associates 
in 1964 and made it a wholly owned subsidiary. SRA established the Center in 
late 1965 to explore computer applications in education (more specifically, in 
instruction, not in educational administration). For three years, the Center 
largely operated as a laboratory where a great deal of intermingling of per- 
sons and ideas took place. Interdisciplinary work was encouraged; there was 
much opportunity to experiment with various ways in which the computer could 
be used in instruction. 

The Center has had four directors since it was started in 1965. Only the 
last two, Capettini and Jeltema, were significantly involved in APTP. However, 
before either Capettini or Jeltema became director, the Center had risen to its 
peak in terms of facilities and personnel (i.e., 35 people) and had already 
begun to decrease in size. At its zenith, the Center had an organizational 
structure consisting of a director and managers of curriculum design, evalua- 
tion, programming, and computer operations. Only three individuals are now 
employed by the Center,*, they are Fouch, Foster, and Jeltema. 

The Center presently markets APTP, Computing Concepts in Mathematics, and 
APL/360 Reference Manual. Almost total emphasis is actually on APTP. Accord- 
ing to Jeltema, 

There were several other things which were left in a pre- 
publication form. In fact, there must be close to 25 or 
30 monographs on applications of the computer at the 
senior high and college level in the fields of mathematics, 
chemistry, and physics. 

Original Development Plan 

Initial work on a formal development plan for APTP occurred in late 1967, 
with Mr. Fouch (transferred from SRA's Materials of Instruction Division) 
working closely with Arthur Siegel of IBM's Instructional Systems Development 
Division in San Jose, California. The broadest outlines of their planning 
were based on a change in direction of the Center from educational research 
with special purpose computing equipment such as the IBM 1500 System to 
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product development for standard hardware (specifically, the IBM System/360 
and standard typewriter terminals) . 

Educationally, the computational skills of elementary arithmetic were 
selected as being a manageable piece of material — large enough to be of 
significant use to schools and yet small enough that development time would 
not be excessive. These computational skills were to be broken down into 
"molecules" and further broken down into smaller steps called atomic skills, 
and these atomic skills were to be organized in non-linear hierarchies. 
Criterion-referenced testing was to provide the basis for placement of the 
individual student and for advancement within the hierarchy. Individualized 
diagnosis, prescription, and monitoring of skill development was to be the 
aim. 



Modifications in Original Development Plan 

No significant changes were made in the original development plan. 
According to the developers, the program emerged pretty much as planned and 
pretty much as scheduled from 1968 on. Fouch explains that this occurred 
only because the original plan was very broad and flexible, details (such as 
precise scheduling) were filled in only when earlier stages were completed. 
While the various options (e.g., the "M" and "S" options) were not planned 
originally, they are considered extensions of original ideas rather than 
modifications in the original development plan. 

Actual Procedures for Development 

Development 

The development of APTP was through the approach of the interdisciplinary 
team: a group of individuals, each with some highly developed specialty and 

each with at least some knowledge and/or experience in the areas of the other 
specialists. For example, nearly everyone on the project had at least a 
rudimentary ability in computer programming; a very large majority of personnel 
had a background of teaching experience; a variety of specialists had experi- 
ence in development of diverse materials of instruction. The subject matter 
and curriculum design specialist, the computer programming specialists, the 
evaluation specialists, and the management specialist all had to Interact 
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frequently in the development of APTP. Interaction and communication were 
greatly facilitated by these overlapping competencies. 

Essential background to the development of APTP were several signifi- 
cant trends which had begun in the early 1960's and which were becoming common 
practice by this time. The third generation of computers was moving rapidly. 
The time— sharing system not only enabled computers to be programmed to operate 
and store work on several programs at a time but also to work on them in a 
sequence sufficiently rapid to give the user the impression that his work was 
being done at the same time as that of another user. This third generation of 
computers also brought along the connection of input-output devices to loca- 
tions remote from the computer's actual processing unit. A person therefore 
dldn* t have to "own" a computer ; he merely needed to buy or rent his own 
input-output device and rent time from the central processing unit, connect- 
ing with it over telephone lines. 

Systems thinking h 2 id been Introduced into the. construction and use of 
curricular materials. Operant conditioning and immediate reinforcement theory 
were coming into their own. These effects were being brought together into 
curriculum construction as teaching programs. Thoughts were broken down into 
definite pieces which were presented for assimilation. Immediate comparisons 
of answers with desired answers were provided, and students were permitted to 
progress if right, and were sent back for additional work if wrong. The 
differentiation of skills into units raised interesting questions about the 
dependence of performance in one unit on mas tery of prior units . This hler- 
archlzatlon of units became a central theoretical issue in the development of 
APTP. Programmed instruction finally brought the matter of criterion- 
referenced testing into the fore. It shifted testing from student- to-standard 
comparison. Additionally, the computer suggested the possibility that test 
questions could be given and scored even before the student was given his 
next question. This opened the possibility for "branching" testing, that is 
for testing which would take the person to places in the universe of content 
closer to where his skill level actually existed. The developers found in the 
universe of arithmetic proficiency content that it therefore became possible 
to Introduce a placement and development sequence into the system programming 
of the Arithmetic Proficiency Training Program. 
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These trends, which were becoming common practice, greatly Influenced 
the original development plan (noted above) that led to the actual develop- 
ment of AFTP. 

The developers, especially Fouch, had looked over many of the numerous 
applications of the computer in instruction. Computer-assisted instruction 
(CAI) was becoming quite popular at the time, but it was noted that many CAI 
programs were little more than linearly programmed textbooks Inserted into a 
computer system, and that the only Individualization that resulted was in 
slightly differing rates of going through the same sequence of material. Such 
programs seemed very unsatisfactory, both in terms of improved instructional 
effectiveness and cost-effectiveness. It was also noticed that most CAI pro- 
grams made only limited use of the potential of the computer (particularly 
for branching and decision-making) and this was in part due to seeming con- 
straints of special-purpose CAI languages. 

If. was therefore decided to start with educational objectives (in general, 
maximum Individualization, immediate reinforcement, high motivation) and to 
find ways to make the computer do the things needed to achieve those objectives, 
even though they might not have been done in other CAI efforts. The choice of 
a computer language became very Important at this point; the language chosen 
was AFL/360, a language that had already been used with considerable expertise 
by CRIS programmers. AFL is a general purpose, high level interactive comput- 
ing language. The "high level" nature of AFL means that it is very easy for a 
programmer to write programs of considerable power quickly, to debug and to 
revise them quickly; this is in contrast to lower level languages which may 
require the programmer to spend large amounts of time on minute technical 
details. The developers thus gained the advantage of being able to try out 
new ideas and sophisticated techniques very quickly, and also to make quick 
revisions. It should be mentioned that, on the other side of the balance 
sheet, AFL has the disadvantage of requiring a rather large computer and, 
as a consequence of this, AFTF was eventually reprogrammed in Coursewrlter III, 
a language demanding much greater effort from programmers but requiring an 
appreciably smaller computer. 

The developers see the development of AFTF as occurring in three phases: 

(I) informal tryout; (II) development and testing of the AFL version; (III) 
reprogramming in Coursewrlter III (the final product version). Together, the 
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three phases constituted a sort of evolutionary process. In Phase I, segments 
of the program with many different variations were tried out with students who 
came Into the Center voluntarily after their regular school hours. Many Ideas 
were discarded as a result of this and many new ideas came out of watching these 
students and talking with them about their reactions. 

Phase II began in the spring of 1968 and consisted of the preparation of 
the APL version to be tested in the Chicago Public Schools during the school 
year 1968-69. Still taking advantage of the fast revision potential of APL, 
many changes were made during the testing process as the feedback from student 
use would show inadequacies in the first draft and would produce m w Ideas for 
Improvement. 

Phase III began in early 1969 and considerably overlapped Phase II. When 
satisfaction had been attained about the early segments of the APL version, 
reprogramming of these segments in Coursewriter III would begin while testing 
of later segments of the APL version continued. Although it is largely true 
that the completed APL version might be said to constitute the detailed specifi- 
cations for the Coursewriter version and that the two versions were intended 
to be indistinguishable to the teacher or student, It Is also true that further 
modifications and Improvements were built into the program during Phase III. 
These were largely enhancements to the program (e.g., the "M" and "S" options), 
things which the Chicago use had shown to be desirable although not necessary. 

In addition to the formative evaluation of the program in the Chicago 
Public Schools, the Comp>uter Related Instructional Systems Center made arrange- 
ments with the Memphis, Tennessee Public School System to field test the pro- 
gram. This Is described under Summatlve Evaluation. 

Since the Arithmetic Proficiency Training Program Is essentially a computer 
program licensed for operation on an IBM System 360, the program had to pass 
IBM's systems standard test before It could be released and licensed. This is 
a technical test to Insure that the program will operate according to specifi- 
cations when leased to a user. This is a standard policy within IBM and 
therefore within Science Research Associates, an IBM subsidiary. The program 
was subjected to these tests during 1969, in conjunction with the IBM Service 
Bureau Corporation and an IBM group In Poughkeepsie, Mew York. The program 
met standards with only slight modification to compensate for too high density 
coding In a few places in the program. 



See Figure 6, the Major Event Flow Chart, for the main activities lead- 
ing up to, including, and following the actual development of APTP. As 
iiidicated in the chart, development took only about a year and progressed 
rather rapidly once the basic concept of APTP was clear to the developers. 

Formative Evaluation 

Beginning in October 1968, the Computer Related Instructional Systems 
Center and the Chicago Public Schools entered upon a field trial of the 
Arithmetic Proficiency Training Program in grades 6, 7, and 8. Two hundred 
ninety-three students in the program who were attending one of the eight 
schools were initially Included in the sample. Six of the eight schools 
sampled were from the inner-city of Chicago. Forty-two percent of the students 
were black, 47% white, and 11% other (primarily Spanish-speaking). Attrition 
occurred but substitution of students was arranged when necessary and possible. 
Two hundred twenty-six students participated fairly regularly throughout the 
program’s trial. Two typewriter terminals were installed in each school and 
connected by telephone lines to a remotely located time— sharing computer 
system. The program, the terminals, and the computer time were all supplied 
by the Computer Related Instructional Systems Center as was the evaluation 
staff and report. Students were scheduled daily for 20-minute sessions with 
the program. In the course of the seven-month study, daily performance of 
the students was recorded; subjective opinions of students, teachers, adminis- 
trators , and parents were assembled; and final achievement results were obtained 
for the Involved students and a comparison group. 

An example of the individualization achieved by the program is in the 
molecule on subtraction of whole numbers. One student worked 16 problems in 
a period of 9 minutes, 40 seconds, to demonstrate his mastery of the highest 
level atomic skill in the molecule. The low end of the range was held by 
another student who worked 1,332 problems in a period of 15 hours, 40 minutes, 
to demonstrate the same level of skill mastery. Records like these were 
readily secured from the program since student reports were generated at the 
conclusion of each session. Assembly and analysis of these reports provided 
specific Indication of the track which the program and the student jointly 
took for the student to reach mastery in each of the molecules assigned or 
attempted. Such records demonstrated that the majority of participating 
students did make significant progress. 



Figure 6 
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Opinion data were secured in May 1969 from students, teachers, administra- 
tors, and parents. A questionnaire and an interview were used. Findings 
included : 



. High degree of sustained motivation in students — In May, 94% of the 
students reported that they liked the program; 75% reported they liked 
it more at the end of the year than they did at first; teachers 
unanimously reported that the students were more enthusiastic. 

. Positive influence on student performance in arithmetic — 82% of the 
students reported that the program helped with their work in their 
mathematics classes; 81% reported that they liked arithmetic in general 
more th 2 in before and that it was more interesting to them; 79% of the 
students reported that they worked arithmetic problems more carefully 
and faster than they did before participating in the program; 82% of 
the teachers noted Increased student Interest in mathematics and Improved 
student performance in regular ma'thematlcs classes. 

. Immediate feedback creates Increased student Interest — 88% of the 
students reported that they were able to tell immediately the quality 
of their dally work at the terminal; students liked immediate feedback, 
particularly in the case of correct responses — 96% of the students 
reported that they liked to be told they were correct; 64% reported 
that they liked to know immediately when they were wrong. 

. Observation of student behavior shows desire to participate In A.P.T.P — 
Observation indicated that students tended to arrive early for A.P.T.P 
sessions, and there was no evidence of cutting sessions though students 
were "on their honor" to go to the terminal room. 90% of the principals 
and 100% of the teachers reporting stated that there was little or no 
disruption of classrooms or hallways due to student participation in 
A.P.T.P.; many students not in the program asked to be included. 

. Parents' reactions favorable — Teachers and principals reported that the 
attitude of parents they came into contact with was generally favorable 
to the program; many parents whose children were not in the program 
requested that they be Included whenever an opportunity presented Itself. 

. Teachers' attitudes generally favorable — Teachers involved in the program 
unanimously agreed that there was no disruption of normal classroom 
activity due to children participating in A.P.T.P.; they also unanimously 
agreed that they would like the opportunity of using other CAI materials, 
thus seemingly indicating that their initial experience with a CAI 
program was a good one; also, 77% of the teachers reporting felt that 
all students would benefit from using A.P.T.P. 



(Rehula, et al 1969) 
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Students were pretested (October 1968) and posttested (May 1969) on the 
Computation Subtest of the SRA Achievement Series and on another test 
especially prepared for the educational objectives of the Arithmetic Pro- 
ficiency Training Program. On the standardized Computation Subtest, the 
adjusted mean scores for the program group differed significantly from that 
of the comparison group In the sixth grade subjects, but not In the seventh 
and eighth grade subjects. The proportion of whole number problems contained 
In the Computation Subtest diminishes from grade six through grade eight; and 
since the majority of students using the program practiced for the most part 
In Its whole nuinber section (only 357 » progressed beyond whole numbers) , the 
effect of practice from the program might not have shown up well on the 
Computation Sub test. In the sixth grade group. In particular, results also 
Indicated that the program took its students from below national norms in the 
beginning to well above them by the end. The comparison group remained below 
these norms In both testings. On the specially constructed test, the program 
group did significantly better on its facts section than did the comparison 
group. The difference did not hold up In the total computation score on this 
special test. However, when computation with whole numbers was separated from 
the total computation score, the difference between the program and comparison 
groups was not significant once again. 

Although the pilot test of the Arithmetic Proficiency Training Program In 
the Chicago Public Schools yielded the overall data summarized above, a 
specific purpose of the pilot test was to gain information on the operating 
characteristics of the program Itself. A number of observations on those 
Initial operating characteristics were especially Informative for understand- 
ing the problems Involved In automating an Interaction between student and 
material in the computational area. 

Equipment down time . Out of a possible 745 hours of operating time, the 
computer was down 35 hours, 43 minutes. This was about 5% down time. Out of 
1,625 usage hours at the 16 terminals, 450, or 5.2%, proved unavailable for 
use when wanted. These data are for the AFL version; the developers state 
that later data for the Coursewrlter version has Indicated considerably 
improved reliability. 

Molecule size . The time needed for students to complete a whole number 
molecule was underestimated. Particular problems were encountered with the 
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difficulty level of upper -numbered skills 
These problems were temporarily solved by 

cule vdille the number of skills In each of the molecules was then shortened | 

i 

to prevent similar difficulties again. | 



Placement phase . A small test was made to determine If students could I 

pass the test which the program skips over as it works to get the student i 

placed in the molecule. Results were affirmative In this trial. Placement | 

worked. | 

Test mode . It was found that the diagnostic test accurately predicted | 

that a student did not need practice when he passed the test. However, the j 



necessity for practice when a student failed the diagnostic test did not emerge 
as clearly. It was hypothesized that the work on problems In the test mode 
actually served as practice which carried over Into the developmental aspects 
of the program which occur some time after a diagnostic test Is failed. 

Practice mode . Eighty percent accurate responses seemed a reasonable 
criterion of expected accomplishment. Typically, students passing a skill 
In test mode passed the experimentally arranged subsequent practice sessions 
on that skill within one problem block. 

Developmental phase . Developmental work In the program does result In 
generalization to problem solving In the future. Results of this assumption 
were reported earlier on the standardized and special tests. 

Timing . Students liked the timing competition which the program arranges. 
Nevertheless the accuracy objectives of the program proved to be met more | 

readily than the timing objectives. The timing objectives were therefore j 

considerably modified after this field experience. But there were still ques- | 

J 

tlons which needed to be addressed by studying the relationship between | 

accuracy and speed In the acquisition of computational skill. j 

! 

Keyboard training . The molecule on keyboard training seemed to work I 

effectively. However, the keyboard Introduced difficulty In program opera- 1 

} 

tlon In early sessions for a student, particularly his first session. It 

took a second or third session for students to become familiar with the key- ' 

board and problem formats cuid responses. 

Readlblllty of program messages In whole numbers . Ninety percent of the 
vocabulary of the program fits Into the Thorndlke-Lorge estimation of grade 5 | 



In the whole number molecules, 
placing students on another mole- 
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vocabulary. Reading level for messages as entire articles was grade 4.8. 

The grade level of messages, therefore, had to be lowered In several regards 

to set the lower grade levels for which the program Is recommended. I 

The terminal . The typewriter In general proved an effective medium for i 

program presentation to the student and for student response. However, a 
number of needed engineering changes were unearthed during the course of the 
pilot testing. The plastic shield under the typeball and the roller bar were i 

necessary for operation, but disconcerting during problem solving when the 
sheet had to be closely watched for alignment. Difficulties were experienced i 

with problem alignment and with the needed double back spacing of the type* 
writer to achieve column alignments In solving a problem from right to left. i 

Some reprogramming had to be done In both Instances. The height and angle of | 

the terminal also caused difficulty for some students. An adjustable chair ^ 

proved necessary. Two-part paper caused difficulty because paper shifted. i 

One-part paper worked better. Some need for reconsidering the typewriter was i 

advanced but the advantages were still felt to be sufficient to keep It In use. 

The computer and the program . Time for the computer to respond to a ' 

student's answer to a problem proved to be a considerable factor In maintain- 
ing student Interest. It was recommended that this should be kept to a maximum 
of five seconds. (The developers say that these excessive computer response 
times were a result of Inefficient programming of the AFL version and of an 
overly heavy load on the computer. With the Coursewrlter version, the goal 
has been a maximum response time of 2 seconds and this has generally been 
achieved. Only of the computer down time was directly attributable to the 
program. This percentage was called "tolerable," but It obviously called for 
understanding from users and capable operators at the central processing unit 
of the program. 

A number of changes were made In the Arithmetic Proficiency Training 
Program as a result of Its Initial field trial In the Chicago Public Schools. 

Reported modifications Included: 

1. The keyboard program was modified to Include further Instruction. 

2. Automatic positioning of the typeball to Its desired first 
position was provided for each problem to be solved. 

3. Programming was Included to guard against a student having 
to return to the very beginning of his session each day if 
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the system "goes down" during his session. Programming was 
Instituted which permitted him to pick up at some Inter- 
mediate point from the beginning when the system started to 
function again. 

4. Several new options were made available to the teacher or 
supervisor of the program. These enhancements were achieved 
by making use of certain aspects of the Coursewrlter language — 
things that were not feasible In the APL version. 

5. Messages to the student were shortened to make the program 
more readable. 

6. A worked out example was Inserted at the beginning of each 
new skill in order to show the student the format of the 
answer In the skill. 

7. Several procedures formerly performed "by hand" were automated. 
These Included the movement of a student from one molecule to 
the next. 



SUMMATIVE EVALUATION 

The Computer Related Instructional Systems Center was In conversation 
with the Memphis Community Learning- Laboratory during the same time that Its 
first formative evaluation with the Chicago Public Schools was In progress. 

Out of these conversations came the first user operated trial of the Arithmetic 
Proficiency Training Program. 

Prior to the Memphis field test, a first draft of the reprogramming of 
the Arithmetic Proficiency Training Program into the Coursewrlter III language 
had been completed. Coursewrlter III had then been accepted as the IBM compu- 
ter language for computer-assisted Instruction; this reprogramming also 
Included subroutines programmed In Assembly Language. This combination of 
languages gave the program greater versatility by providing additional teacher 
directed functions which were hard to permit In APL, the language In which the 
program was originally written. However, using both languages for the program 
gave the entire program greater flexibility and power than would have been 
available In either language alone. 

Lea W. Joyner, CAI Coordinator In Memphis, and Longno A. Cooke, Jr., 
Director of the Memphis Community Learning Laboratory, were primarily respon- 
sible for the conduct and report of the Memphis field test of the Arithmetic 
Proficiency Training Program. As will be noted, this field test was a small 
scale effort. 
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The experimental group to which the Arithmetic Proficiency Training 
Program was given in this 1969-70 field test consisted of 15 sixth grade 
students. The experimental group was matched by 15 other sixth grade pupils. 
The Thorndike-Lorge IQ Score as of November 1968 and the April 1969 Arithmetic 
Computation Score on the Metropolitan Achievement Test were used simultaneously 
for this matching. The matching also resulted in acceptance of the hypothesis 
that the experimental and control groups did not differ in their Arithmetic 
Problem Solving and Concepts scores on the April 1969 Metropolitan Achievement 
Test. 

On a different form of the Metropolitan Achievement Test administered to 
both groups in' April 1970, the experimental group performed significantly 
better than the control group as indicated by their Computations, Problem 

Solving, and Concepts scores. It did appear that the 16 to 24 hours which 
the experimental group had on the Arithmetic Proficiency Training Program 

resulted in achievement gains. No significant correlations were found relating 
amount of growth to either IQ or length of time on the program. 

DIFFUSION 

Agency Participation 

I*iiti®lly» SRA staff associates were given primary responsibility for 
diffusion activities and marketing. Recently, all aspects of diffusion and 
marketing have become concentrated in the Computer Related Instructional 
Systems Center. Marketing APTP is a major function of the Center at SRA. 

Diffusion Strategy 

The strategy was essentially the same as that used by SRA for its other 
products: promotion by SRA's sales force of staff associates, by direct 

mailings, by catalog listings, etc. However, this was complicated by two 
factors: the school’s need for computer equipment capable of running CAI 
programs (which they seldom had prior to their interest in APTP) , and the 
need of marketing personnel for a great deal of technical knowledge about 
computers. APTP was sold only by SRA, but it ran only on IBM equipment which 
was not sold (or leased) by SRA. 
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A further complication resulted from IBM's long-standing practice of 
providing services and programs with the sale or lease of its equipment. 
However, in June 1969, IBM announced its decision to price services and pro- 
grams separately from equipment. This supplied a simple solution to SRA's 
problem of following general IBM practices and, at the same time, of gaining 
revenue from CAI programs. It was therefore decided to price AFTF in a 
fashion parallel to IBM's new practices for leasing program products. 

Actual Diffusion Efforts 

The sale of the Arithmetic Froflclency Training Frogram represented a 
marketing "first" for Science Research Associates. SRA was going out to sell 
materials involving the computer as a medium. SRA therefore put a lot of 
attention and effort into marketing the Arithmetic Froflclency Training Fro- 
gram. Internal announcement of the program's availability was scheduled for 
mid-September 1969. A selected group of SRA's staff associates for selling 
its materials to elementary and high schools was brought into SRA's Chicago 
home office for a five-day conference in August 1969. Since most of these 
staff associates were unfamiliar with computers, they were given three days 
of training about the computer and its programming. Materials used Included 
Computing Concepts in Mathematics , thus also familiarizing them with another 
product of the Computer Related Instructional Systems Center. The last two 
days of this conference were spent training the staff associates specifically 
about AFTF. The Arithmetic Froflclency Training Frogram was unveiled a month 
later. Sales people were urged to sell it. 

Public announcement of the forthcoming availability of the Arithmetic 
Froflclency Training Program signalled a second step in the program's market- 
ing effort. Personnel for the Computer Related Instructional Systems Center 
hit the "sawdust trail." Demonstrations of the Arithmetic Froflclency Training 
Frogram were arranged in several major cities. SRA staff associates were host 
to their contacts in these cities. The Center came in and made its "pitch." 
Problems were encountered in securing terminals. Jeltema reported that there 
just wasn't any effective substitute for the "hands on" experience with the 
Arithmetic Proficiency Training Frogram. When an Interested customer allows 
himself to Interact with the program, the program begins selling Itself. Until 
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that time the program is an Interesting "toy" of our technological age. 

Jeltema noted: 

As things evolved, we found that one of the best ways to 
demonstrate is to see It In operation In a school. Another 
excellent way Is to get a terminal and capture a child 
who's had no previous experience but gets about five minutes 
of how to do It. It's amazing how well they operate even 
with that kind of background. Still another effective way 
Is to have the person sit down and do It himself. The least 
effective way Is either to do It for him or talk about It 
without the terminal. 

At the present time the Center Is experimenting with videotape presenta- 
tions. The Center Is also beginning to push Its recent Incorporation into 
the program, the spiral curriculum. 

A brochure explaining the system Is available and distributed widely — 

"ir the thousands." The brochure has a return postcard by which a person 
expresses Interest and requests further help. These postcards are followed 
up In various ways. 

In the beginning. Interest was passed along to SRA staff associates who 
were entrusted with following It up. Staff associates sell on a commission 
basis. Sale of the Arithmetic Proficiency Training Program was put on this 
comoilsslon basis at first. However, It quickly became apparent that the 
Installation of the program Is complex and that sale Is neither easy nor 
frequent. For Instance, Installation Includes: training the teachers; training 

the proctors; acting as educational counselors; helping the buyer design and 
experiment for evaluation; and being present when children operate terminals 
for the first time. In addition, a school system has to have or rent computing 
facilities and terminals. All this means that the school has to be quite 
Interested In the program before It Is willing to take on the program's added 
effort and cost In the school's mathematics Instruction. In addition, the 
Center has been plagued by the schoolman's need for proof "that it works" 
before he feels able to go to his school board and taxpayers with a request 
for the needed funds and support. As is Indicated above, field trials do 
Indicate that the Arithmetic Proficiency Training Program "works." However, 

It Is also evident that how well and with whom It works has not been convincingly 
demonstrated. The logic of the program's operation has proven Insufficient to 
sell widely. 

34 
O 

ERIC 

^ 



40 



w 



This experience led the Computer Related Instructional Systems Center to 
withdraw marketing of the Arithmetic Proficiency Training Program. from general 
responsibility of SRA staff associates. The result is that marketing is now 
concentrated in the Center Itself: "We met our objectives in terms of sales, 

and obviously they were modest. We didn't expect it to get started quickly. 

We do expect it to grow." 

Product Characteristics and Other Factors Affecting Diffusion 

APTP was designed to be a supplementary mathematics program that would 
be compatible with most school practices. The computer programs that make up 
the software can be leased as a complete unit. Teacher training Is provided, 
but only through the teacher's guide and the consulting services of SRA. 

Start up costs, as noted earlier, are often considered expensive given the 
economic conditions of today, the need for computer facilities, and the fact 
that APTP is a supplementary program. 

ADOPTION 

Extent of Product Use 

Extent of use of a CAI program is viewed by the developers as a difficult 
thing to measure by any single number such as number of students (since this 
does not show anything about how much use was made by each student). They favor 
stating the number of terminals dedicated to CAI use, but admit that this is 
not satisfactory either since it falls to show whether a terminal was used for 
one hour or for 100 hours each week. 

In the four school systems using the program after the formative evalua- 
tion in Chicago, the following numbers of terminals were Installed: 



Oakland, Calif. 


(Spring, 1970) 


35 


Memphis, Tenn. 


(1969-70) 


6 




(1970-71) 


16 


Tacoma, Wash. 


(1971) 


15 


Newark, N. J. 


(1971) 


15 



APTP may be used in sessions of any length specified by the teacher; 
however, the developers do not recommend sessions of less than 10 minutes. 

A better picture of the extent of use may be acquired by imagining sessions 
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averaging 15 minutes each^ during a slx—hour school day< Thus, each terminal 
24 students per day» This number can be Increased considerably by 
extending the use of the program to after-school hours, to adult education In 
evening hours and on Saturdays, etc. The situation Is further complicated by 
possible variations In scheduling Individual students: each student might be 

scheduled for a session every day or every other day or two days a week; and 
this scheduling might be for a whole school year, for six weeks, or until the 
student has demonstrated a certain achievement level. 

Memphis, Tennessee represented a sale which was a continuation of its 
field involvement. Newark and Tacoma represented new sales. However, the 
first new smle was to Oakland, California schools during spring 1970. The 
Oakland schools came in fast because they had the available computer facilities 

1“ the education of disadvantaged pupils. However, they did 
not continue to buy %»hen initial supplementary funds were exhausted. 

AFTP was also recently used by an IBM manufacturing plant in Manassas, 
Virginia. The program was used initially in the Manassas plant for a small 
ezperlisent in training a group of new-hires in a three-month training program 
in which the plant participated under the National Alliance of Businessmen. 

The Mfinassaa experience led the plant’s management to decide to Include the 
progran In their ongoing voluntary employee education efforts. Several IBM 

sites are expecting to use the program for their employee training programs. 

% 

Installation Procedures 

AFTP requires a computer and terminals, and the necessary room organization 
student to work at the terminal when supplementary mathematics 
work is scheduled. Some public relations effort prior to adoption is advisable 
and administrative support is needed. The latest options allow the teacher to 
modify the program to meet her needs. Teacher training is important. The 
developers feel that CAI is a new tool for use in the instructional process 
and thatf like any other tool, it will produce valuable results only when 
used by people who understand it and who have become skilled in its uses. CAI 
is not a tool that can really run all by itself nor is it so ideally designed 
as to be incapable of misuse. Until some future time when schools of education 
regularly give courses in the use of CAI, the individual school users of CAI 
will meed to supply the needed special training, probably with the aid of the 



development group. As noted earlier, and below (see Table 1), in the suggested 
organization of personnel, there are extra staff requirements. 

SRA has prepared a suggested organization of personnel for schools that 
adopt APTP. They have developed a chart that lists the major tasks to be per- 
formed and indicates with an X the person best suited to perform each task. 
Where more than one X appears, anyone of the persons indicated may suitably 
perform the task. See Table 1. As described in the chart, the systems super- 
visor is the overall coordinator of the program. The proctor monitors the 
terminal, handles any problems having to do with the functioning of the program 
or equipment, and takes responsibility for the students using the terminal room. 
The data processing manager is responsible for the functioning of the computer 
and for maintenance of the machine-readable materials. The developers point 
out that this is only one of many possible organizational charts. In some 
schools, it may be more advantageous to assign tasks to persons other chan 
those indicated in the chart. The method of organization a school chooses will 
depend on the talents and availability of its personnel. 

FUTURE OF THE PRODUCT 

The Arithmetic Proficiency Training Program emerged as one of three pre- 
sent products of a developmental effort organized in late 1965 within Science 
Research Associates to develop computer related instructional materials. 
Mathematics programs are two of the Center's three presently available products. 
An APL manual is the third. In actuality, almost total emphasis in the Center 
is on APTP. 

The effectiveness of APTP was suggested, but not convincingly demonstrated, 
in several field trials. Present customers find the product useful. IBM 
educational programs are beginning to use the program, suggesting that it is 
cost effective as well. But still a "million dollar" product is being used 
by no more than eight schools. There is no obvious single answer to why this 
is so, but some of the factors noted under Factors Affecting Diffusion suggest 
possible reasons. APTP requires a central processing unit of the size of the 
IBM System/360 Model 30. The program requires installation, operation, and 
servicing terminals. The program requires that teachers adapt themselves and 
their instruction. The program requires a proctor. Thus, there are the usual 
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Table 1. Suggested Organization of Personnel 


\ 


Task 


Proctor 


Teacher 


Computer 

Operator 


Data 

Processing 

Manager 


APTP 
Sys terns 
Supervisor 


Select students 




X 






j 

! 


Schedule students 




X 






J 

X i 


Give students a brief introduction to 
the pedagogy of the program 




X 






i 

1 

i 


Determine modification of the pro- 
gram for individual students 




X 






\ 

X 1 


Give students needed arithmetic 
instruction after a teacher call 




X 






1 

i 

j 


Determine modifications of the pro- 
gram that are necessary for indi- 
vidual students when they receive 
a teacher call 




X 






j 

A 

1 

_ ] 


Enter program modifications into 
the computer 


X 








% 

V 

■i 

■1 


Monitor terminals 


X 








>1 


Keep all necessary external APT? 
records 


X 








1 

■•I 

3 


File and maintain all necessary stu- 
dent material and equipment 


X 






, 


i 

.4 

1 


Order supplies 


X 










Call repair service for terminals | 

or data-phones I 


X 








1 

t 

i 


Register students | 






X 


X 


X i 


Monitor computer | 






X 




J 

n 


Give students brief Introduction 1 

to the use of the terminal | 




X 






1 

■} 


Answer proctor questions per- 1 
talnlng to the terminal or 1 
computer 1 








X 


i 

X 1 


Obtain answers to questions asked 
by the proctor related to other 
than terminal or computer 






a 




. 1 


Authorize repair service calls U 








X 


X 


Serve as liaison between proctor 1 
and other staff members involved H 
in the use of APTP | 










X 


Maintain machine readable materials | 






X 


X 


i 
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obstacles to school improvement; namely, added equipment, noise, nuisance, 
understanding, and personnel. But the Arithmetic Proficiency Training Program 
suffers an additional and very large obstacle as well. It takes a time*~shared 
computer to service the interactions of the program. Not many schools command 
computers or their time. Thus the Aritlunetic Proficiency Training Program 
suffers as a product without a delivery network. At the same time, sale or 
rental of computers in schools suffers from interest in a new delivery system 
which lacks a sufficient quantity of sensible materials like the Arithmetic 
Proficiency Training Program. It remains to be seen how SRA and IBM will 
jointly approach its continuing problem of developing more educational soft” 
ware that will be adopted by schools. 



CRITICAL DECISIONS 

The following events are a good approximation of crucial decisions which 
were made in the four-year history of the Arithmetic Proficiency Training Program. 
For each decision point, the following types of information were usually des- 
cribed; the decision that had to be made, the alternatives available, the 
alternative chosen, the forces leading up to choosing a particular alternative, 
and the consequences resulting from choosing an alternative. 

Although an attempt has been made to present the critical decisions or 
turning points in chronological order, it must be clearly pointed out that these 
decisions were not usually made at one point in time, nor did they necessarily 
lead to the next decision presented in the sequence. Many of the critical 
decisions led to consequences that affected all subsequent decision-making 
processes in some Important way. 

Decision 1; To Analyze Skills into Sub-Skills and to Organize These Skills 
into Hierarchies 

Once the developers had narrowed their forces to arithmetic computation, 
they had to decide how to organize the computational skills. They knew students 
needed to start near their present level of competency and work at their own 
level and at their own pace. On the basis of this understanding, the developers 
decided to analyze the computational skills into sub-skills and then to organize 
these skills into hierarchies of prerequisites. Thus the student could start 
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near his present level of competency and proceed by small steps to mastery of 
the most complex skills that he needs. Tltis decision was very much in line 
with the trends of the time. 

Decision 2; To Use the Computer to Test, Place and Develop the Skills in 
Students . 

The Computer Related Instructional Systems Center was given the assignment 
of figuring out how the computer could best be used to individualize instruc- 
tion in elementary arithmetic. While the decision to use the computer was, 
thus, partly determined, the developers realized that the computer could offer 
the student individualization in both diagnosis and practice. At the same time, 
it could allow the teacher to direct the student's practice without having to 
supervise it. The computer could allow flexibility in decision making, through 
branching programs and through personalized diagnosis and imme'iiate feedback. 
While this decision to use the computer facilitated the incorporation of these 
options considered critical, the decision may also have limited the extent of 
one of the products because of the economic considerations. 

Decision 3; To Have the Develor^rs Work Closely Together 

The developers decided quite early that the specialists in each area (e.g., 
curriculum design, or computer programming) would have to work very closely 
together. The developers observed that typically the computer programaiers are 
off by thmaselves while the other developers are involved in comprehensive 
planning; what usually results is a series of compromises forced tipon these 
planners by the computer programmers. To pr -vent unnecessary conpromises and 
to benefit from all the potentials of the computer, frequent interaction was 
a general policy that was appreciated by all the specialists. This decision 
facilitated development and helped the developers to tap many of the resources 
of the computer. 

Decision 4; To Pilo-. Test the Program 

Once the program was in operating condition, the developers decided that 
they needed information on how it would work in a real school situation. The 
developers figured that information on the operating characteristics of the 
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program would help solve some of the unrecognized problems involved in auto- 
mating an interaction between student and material in the computational area* 
Their pilot test provided information on such characteristics as: equipment 

down tine, molecule size, timing, placement, and student-terminal interaction. 
As a result of the pilot test, a number of critical changes were made in the 
program* Such changes as the incorporation of the Coursewriter Language 
enabled new options, like the spiral curriculum, to be offered. 
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APPENDIX A 



LIST OF PRODUCTS AND DEVELOPERS 

The following is a list of products for which Product Development Reports 
have been prepared. 

Arithmetic Proficiency Training Program (APTP) 

Developer: Science Research Associates, Inc. 

The Creative Learning Group Drug Education Prcgram 
Developer: The Creative Learning Group 

Cambridge, Massachusetts 

The Cluster Concept Program 
Developer: The University of Maryland, 

Industrial Education Department 

Developmental Economic Education Program (DEEP) 

Developer: Joint Council on Economic Education 

Distar Instructional System 

Developer: Siegfried Engelmann & Associates 

Facilitating Inquiry in the Classroom 

Developer: Northwest Regional Educational Laboratory 

First Year Communication Skills Program 
Developer: Southwest Regional Laboratory for 

Educational Research & Development 

The Frostig Program for Perceptual-Motor Development 
Developer: The Marianne Frostig Center of Educational Therapy 

Hawaii English Program 

Developer: The Hawaii State Department of Education 

and The University of Hawaii 

Holt Social Studies Curriculum 

Developer: Carnegie Social Studies Curriculum Development Center, 

Camegie-Nellon University 

Individually Prescribed Instruction — Mathematics (IPI — Math) 
Developer: Learning Research and Development Center, 

University of Pittsburgh 

Intermediate Science Curriculum Study 
Developer: The Florida State University, 

Intermediate Science Curriculum Study Project 

MATCH*-Materials and Activities for Teachers and Children 
Developer: The Children's Museum 

Boston, Massachusetts 
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Program for Learning in Accordance With Needs (PLAN) 

Developer: American Institutes for Research and 

Westinghouse Learning Corporation 

Science — A Process Approach 

Developer: American Association for the Advancement of Science 

Science Curriculum Improvement Study 

Developer: Science Curriculum Improvement Study Project 

University of California, Berkeley 

Sesame Street 

Developer: Children's Television Workshop 

The Sullivan Reading Program 

Developer: Sullivan Associates 

Menlo Park, California 

The Taba Social Studies Curriculum 

Developer: The Taba Social Studies Curriculum Project 

San Francisco State College 

The Talking Typewriter or 

The Edison Responsive Environment Learning System 

Developer: Thomas A. Edison Laboratory, 

a Subsidiary of McGraw Edison Company 

Variable Modular Scheduling Via Computer 

Developer: Stanford University and 

Educational Coordinates, Inc. 
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